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BRIEF  DESCRIPTION  OF  PORTFOLIO: 

Discover  Mathematical  Algorithms  and  Develop  Computational 
Techniques  that  Provide  More  Accurate,  Reliable,  and  Efficient  Algorithms 
for  Modeling  and  Simulation  and  Design  of  Complex  Systems  for  the  U.S. 
Air  Force  with  radical  cost  and  turn-time  improvements. 

LIST  SUB-AREAS  IN  PORTFOLIO: 

^Multi-Scale  Modeling  (  materials,  plasma,  structures,  fluids, 
r  combustion) 

"  lulti-Physics  Modeling  (fluid-structure  interactions,  particle-fluid 
models) 

jy Uncertainty  Quantification 

'  nary  Optimization  and  Control  (Design  Under 


New  Emphasis  on  Scalable  Algorithms  in  All  These  Sub-Areas 
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Goal:  Simulation,  Analysis  and  Design  of  complex  systems  with  radical  cost 
and  turn-time  improvements 

Needs  for  Computational  Math:  Support  for  Accurate,  Reliable,  and  Efficient 
novel,  mathematically  sound,  and  verified  algorithms  for  complex  AF 
applications  such  as  in  plasma,  structures,  combustion,  fluids,  and  so  on. 

AF  Examples:  1)  Simulations  of  High  Power  Microwaves  (Dr.  Luginsland) 
-  Need  for  3D,  high-fidelity,  parallel  modeling  of  high  energy  density  fields  and 
particles  in  complex  geometry  —  AFOSR  comp  math  support 

2)  “Digital  Twin”  (Dr.  Stargel):  Real  Time,  High-Fidelity  Operational  Decisions 
for  Individual  Aircraft  Enabled  by  Tail  Number  Health  Awareness  :  Need  for 
Material  Scale  modeling,  Deterministic  Multiscale  Modeling,  Uncertainty 
quantification  -  Key  research  areas  in  comp  math  supported  for  several  years. 

3)  Turbulence  Combustion  (Dr.  Li):  Need  for  new  numerical  and 
mathematical  approaches  for  multiscale,  multiphysics  modeling  -  Key  research 
area  in  comp  math 

4)  Aerospace  Materials  for  Extreme  Environments  (Dr.  Sayir):  Need  for 
theoretical  and  computational  tools  that  aid  in  the  discovery  of  new  materials 
and  Mathematics  to  quantify  the  microstructure  —  A  new  research  thrust 
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w  Transformational  Opportunities 
*if  for  Computational  Math 


Predictive  High-Fidelity  Simulations  for  AF  Applications 


•  More  computing  power  based  on  emerging  heterogeneous  computing 
architecture  BUT  computing  power  isn't  everything 

•  Challenges  :  Issues  with  system  Power,  Slower  growth  in  Memory  bandwidth,  Cost 
of  data  movement,  Slower  growth  in  clock  rate  and  move  to  Concurrency  of  higher 
number  of  nodes  and  threads,  Slower  growth  in  I/O  bandwidth,  Resiliency  and  Reliability 
of  the  computing  systems 

We  Need  Fundamental  Paradigm  Shifts  in  Algorithms,  Software, 
_ Architecture  Design _ 

•  2  Basic  Research  Initiatives  in  2012  *  Ultra-scale  Algorithms(with  Dr. 
Luginsland),  and  Co-Design  (with  Drs.  Luginsland,  Smith,  Stargel) 

•  For  Co-Design,  the  emphasis  was  on  integration  of  research  threads  into  a  coherent  agenda  for 
specialized,  high-performance  computing  architectures  for  Air  Force  and  DoD  applications. 

•  For  Ultra-Scale  Algorithms  the  focus  was  on  mathematical  aspects  of  scalable  solvers  with 
emphasis  on  parallelism  across  scales  (spatial  and  temporal),  high-order  discretization,  and 
multi-level  domain  decomposition  techniques. 
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1  &  Portfolio’s  Focus  Areas  and 

yg  Outline  of  Research  Highlights 

*  High-Order  Methods  of  Accuracy(the  sole  funding  agency 
to  support  these  methods  across  various  application 
areas  such  CFD,  Structures,  Plasma,  Combustion) : 

-  An  interplay  of  Efficiency  , Accuracy  and  Robustness —  Serious  need  for 
mathematical  development  of  these  methods  for  capturing  complex  multiscale 
physics  in  an  accurate  AND  efficient  way 

•  Highlight  of  a  CFD  effort 

•  Highlight  of  a  Combustion  effort 

*  Multiscale  Methods:  Materials  —  need  for  more  mathematical  and 
computational  techniques  for  bridging  the  scales 

*  Uncertainty  Quantification:  Need  for  more  mathematics  in  modeling 
and  analysis  of  stochastic  phenomena 

-  Design  Under  Uncertainty --  a  2012  BRI  topic 

-  Bayesian  Framework  for  UQ  ->  a  MURI  2012  in  Material  Science 

-  Scalable  Algorithms  for  Inverse  Problems 
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•  Many  important  problems  require  predictions  of  fluid-structure  interaction  (FSI): 

-  Oscillatory  interactions  in  engineering  systems  (e.g.  aircraft,  turbines,  and  bridges) 
can  lead  to  failure 

-  The  blood  flow  in  arteries  and  artificial  heart  valves  is  highly  dependent  on 
structural  interactions 


•  Two  main  numerical  approaches  for  the  coupling: 

-  Fully  coupled  (monolithic):  Solve  the  fluid/structure  equations  simultaneously. 
Accurate,  but  requires  specialized  codes  and  solvers  are  often  slow. 

-  Weakly  coupled  (partitioned):  Use  standard  solvers  for  fluid/structure  and  apply  a 
separate  coupling  scheme,  often  together  with  sub-iterations.  Efficient  and  simple, 
but  issues  with  accuracy  and  stability. 

•  Approach:  Use  Implicit-Explicit  (IMEX)  schemes  to  obtain  high-order  accuracy  without 
coupled  solvers  or  iterations 


•  Using  high-order  spatial  DG  and  FEM  schemes 
for  the  fluid  and  the  structure,  up  to  5th 
order  of  temporal  accuracy  with  the  IMEX  scheme  is 

obtained 
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Benchmark  Airfoil  System 


High-order  DG  for  Navier-Stokes,  ALE  for  moving  domain 
Rigid  pitching/heaving  NACA  0012  airfoil,  torsional  spring 
Up  to  5th  order  of  convergence  in  time 
Without  the  predictor,  at  most  2nd  order  of  convergence 


t  —  0.0 


I—  =  — k0  —  t  —  IM  cos  0y"(t) 


t  =  1.2 
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V  4  Fluid/Membrane  Interaction 

>  Hyper-elastic  neo-Hookean  membrane  with  viscous,  fluid-like,  dissipation 

>  Membrane  ends  are  held  fixed  but  allowed  to  rotate 

>  Fluid  has  20  degree  angle  of  attack  and  Reynolds  number  2,000 

>  Implicit  schemes  easily  handle  complex  behavior  with  large  time-steps 


DISTRIBUTION  STATEMENT  A  -  Unclassified,  Unlimited  Distribution 


8 


Optimal  numerical  methods  for  chemically  reacting 
flows  will  ultimately  have  to  be  both  locally 
adaptive  and  high  order 

Diffusion  Flame-Vortex  Interaction 

-  Canonical  configuration  facilitating  a  first-principles 
understanding  of  aspects  of  turbulent  combustion  (e.g., 
flame  stretch,  local  extinction,  soot  production) 


*  Simplified  Flame-Vortex  Models 


Hsu,  et  al.,  Phys.  Fluids 
A  (1993) 


—  Convection-diffusion-reaction  equations  with  imposed  flow  field 


—  One-step  or  multi-step  finite-rate  Arrhenius  chemistry 


—  Variable  mechanical  timescale  related  to  a  vortex  Reynolds  number 


—  Benefits  of  simplified  models: 


Faster  development  of  tailored  numerical  methods  for  complex  problems 
Imposing  the  flow  facilitates  joint  theoretical  and  computational  work 
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Flame  Roll  Up  in  a  Vortex  -  Re  =  600 

Conventional  method  under  grid  refinement 


LO,  base  grid,  t=3.00d-4 


-0.3  -0.2  -0.1  0  0.1  0.2  0.3 

Spatial  discretization 

•  Grid:  quasi-uniform,  Ax  «  0.008 

•  Order:  1st  (upwind  scheme) 

•  CPU  factor:  1 


LO,  refined  grid,  t=3.00d-4 


-0.3  -0.2  -0.1  0  0.1  0.2  0.3 

Spatial  discretization 

•  Grid:  quasi-uniform,  Ax  «  0.004 

•  Order:  1st  (upwind  scheme) 

•  CPU  factor:  15.3 
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4  i  Flame  Roll  Up  in  a  Vortex  -  Re  =  600 

Conventional  vs.  implicit-compact  method 


LO,  t=3.00d-4 

T  emperature(LO ) 


-0.3  -0.2  -0.1  0  0.1  0.2  0.3 


CS,  t=3.00d-4 

T  einperaturdCS) 


-0.3  -0.2  -0.1  0  0.1  0.2  0.3 


Spatial  discretization 

•  Grid:  quasi-uniform,  Ax  «  0.008 

•  Order:  1st  (upwind  scheme) 

•  CPU  factor:  1 


Spatial  discretization 

•  Grid:  quasi-uniform,  Ax  «  0.008 

•  Order:  4th  -  6th  (compact  scheme) 

•  CPU  factor:  1 .2 
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Flame  Roll  Up  in  a  Vortex 

Re  =  600 


t=0.00d-4 


-0.3  -0.2  -0.1  0  0.1  0.2  0.3 

Spatial  discretization 

•  Grid:  quasi-uniform,  Ax  «  0.008 

•  Order:  1st  (upwind  scheme) 

•  CPU  factor:  1 


t=0.00d-4 


-0.3  -0.2  -0.1  0  0.1  0.2  0.3 

Spatial  discretization 

•  Grid:  quasi-uniform,  Ax  «  0.008 

•  Order:  4th  -  6th  (compact  scheme) 

•  CPU  factor:  1 .2 
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Multiscale  Modeling  and 
Computation  for  Materials 

•  New  research  thrust  -  New  core  projects,  a  MURI  2012  in  Managing  Data 
Complexity  in  Predictive  Material  Science,  and  a  COE  on  Integrated 
Computational  Material  Science  and  Engineering  (ICMSE),  2013  MURI  on 
Peridynamics 

•  ‘‘The  underlying  physical  laws  necessary  for  the  mathematical  theory  of  a  large 
part  of  physics  and  the  whole  of  chemistry  are  thus  completely  known,  and  the 
difficulty  is  only  that  the  exact  application  of  these  laws  leads  to  equations  that 
are  much  too  complicated  to  be  soluble.”  P.  A.M.  Dirac,  Proc.  Royal  Soc. 

123  (1929),  714-733 


•  Dirac’s  quote  true  today,  and  relevant.  Most  new  technologies  of  interest  to 
the  Air  Force  can  be  analyzed  by  equations  known  (in  1929)  to  be  accurate  and 
predictive,  but  we  are  unable  to  do  it:  The  discipline  that  is  needed  most: 
Computational  Mathematics 

•  Approach: 


•  1)  By  evaluating  all  competing  multiscale  methods  on  the  basis  of  rigorous 

error  analysis  and  systematic  numerical  testing 


2)  New  multiscale  approaches  for  design  of  new  materials  -  Objective 
Structures 
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•  Multiscale  computational  methods  for  materials  need  to  be  validated  by 
benchmark  numerical  experiments  designed  and  evaluated  on  the  basis 
of  theoretical  error  analysis. 


•  Theory-based  benchmarking  for  material  failure 


•  Crystal  defects  such  as  grain  boundaries,  cracks,  and  dislocations  play  a 
central  role  in  determining  material  behavior  for  Air  Force  applications. 
The  key  difficulty  in  simulation  is  that  crystal  defects  affect  elastic  fields 
far  beyond  their  immediate  atomistic  neighborhood  and  give  rise  to 
strongly  coupled  multi-scale  systems. 


Computational  Complexity 
^  of  Defects 


•  Identify  the  quantity  of  interest  (material  failure,  strength,  toughness, 
etc.)  and  utilize  multiscale  error  analysis  to  determine  the  complexity 
(number  of  degrees  of  freedom)  to  compute  the  quantity  of  interest  to 
the  required  tolerance. 

•  The  Blended  Quasicontinuum  Energy  is  the  first  atomistic-to-continuum 
energy  for  many-body  interactions  and  multi-lattices  with  a  controllable 
error.  Numerical  results  for  a  microcrack  are  exactly  predicted  by 
theoretical  complexity  estimates. 
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Multiscale  Methods  for  Materials, 

Kaushik  Dayal  (CMU)  &  Richard  D.  James  (UMN) 


Carnegie 

Mellon 

University1' 


■  Approach:  use  the  underlying  invariance*  of 
quantum  mechanics  as  the  main  principle  for 
simplification 

*  Frame-indifference  and  permutation  invariance 

■  Some  current  implementations  of  this  idea 


Objective  molecular  dynamics.  Based  on  a  fundamental 
invariant  manifold  of  the  equations  of  molecular  dynamics. 
TO:  a  method  for  simulation  of  macroscopic,  far-from- 
equilibrium  fluid  flows  of  any  fluid  using  only  a  few 
“simulated  atoms”.  TO:  a  method  for  simulation  of 
nonequilibrium  properties  of  nanostructures 

Objective  DFT:  An  advanced  DFT  method  for  nonperiodic 
nanostructures.  TO:  predict  collective  properties 
(ferromagnetism,  ferroelectricity,  flexoelectricity,  etc.) 
of  nanostructures  from  first  principles 

Objective  self-assembly:  TO:  predictive  design  of 
functional  macromolecules  to  enable  tailored  self-assembly 
of  nanostruct,,rQO 


Uncertainty  Quantification  Efforts  - 
MURI  ‘09  and  Core  Projects 


Uncertainty  Quantification  (UQ):  The  process  of  quantifying  uncertainties 
associated  with  model  calculations  of  true,  physical  Quantities  of  interest  (QOIs)  with 
the  goal  of  accounting  for  all  sources  of  uncertainties  and  quantifying  the  contributions 

of  specific  resources  to  the  overall  uncertainty.  From  “  Assessing  the  Reliability  of  Complex 
Models”-  NRC  report  in  2012 

>MURI  Effort:  To  develop  rigorous  theory,  algorithms  and  software  for  UQ 

management  in  a  computationally  scalable  way.  -  A  unique  effort  in  terms  of  its  breadth 
of  topics  and  approaches,  and  its  impact  on  the  field. 

>  5  Research  Thrusts: 

>  Mathematical  Analysis  and  Multiscale  Formulation  of  SPDEs 

>  Numerical  Solution  of  SPDEs 

>  Reduced-Order  Modeling 

> Design  &  Control  Under  Uncertainty* 


Research  Highlights 


>  Mathematical  Theory:  Quantization-renormalization  of  SPDEs;  New 
evolution  equations  for  joint-pdf  of  SPDEs;  Nonlinear  Malliavin  calculus 
>Reduced  Basis  Methods  (RBM):  Integral  equations  and  multi-scattering 
problems;  Robust  design,  parameter  estimation,  and  model  uncertainty 

>  Adaptive  ANOVA:  Convergence  theory;  Parameter  compression  and  RBM; 
Fluid  flows,  porous  media,  multi-scattering 

>  Bayesian  Framework:  Coarse-graining;  Active  learning  +  SPDEs; 
Adaptive  SMC,  dependent  random  variables,  Model  uncertainty  in  inverse 
problems 

> Design  Under  Uncertainty:  Multi-Fidelity  approaches 

>Numerical  SPDEs:  Data-driven  stochastic  multiscale  method,  Multiscale 
multilevel  MC,  Probabilistic  graphical  models,  Long-time  integrators  of 
SPDEs 

>Software:  MEPCM  library;  Reduced  basis  method  libraries  -  RBOOMIT, 
RBAppMIT;  Akselos,  Inc;  Random  poly-crystals  -  RPCrystal 
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Multifidelity  approaches:  How  should  we  best  use  all  available  models 
and  data  in  concert  to  achieve: 

Better  decision-making  (optimization,  control,  design,  policy) 

Better  understanding  of  modeling  limitations 


Projection-based  reduced  models 

•  Exploit  problem  structure 

•  Embody  underlying  physics 


Simplified 
physics  models 


Research  objectives:  Develop  and  apply  new  multifidelity  methods  for 
design  under  uncertainty 


Multifidelity  model  construction  (certified  reduced  basis  models) 

Quantification  of  uncertainty  and  model  fidelity  (how  good  is  a  model  for 
a  given  purpose) 

Multifidelity  model  management  (which  model  to  use  when,  convergence 
guarantees,  model  adaptation,  model-data  fusion) 


19 
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Multifidelity  Optimization  under  $ 

Uncertainty 


AEROASTRO 
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Traditional  OUU  methods 

•  Require  thousands  of  high- 
fidelity  model  evaluations 
at  each  optimization 
iteration 


hi-fi  model 


Adaptive  corrections:  Exploit 
model  local  accuracy 

•  Computed  using 
occasional  recourse  to  high- 
fidelity  model 

Surrogates  constructed  to  have  desirable  properties  (e.g.,  for  convergence)  ^ 


Control  variates:  Exploit 
correlation  between  high- 
and  low-fidelity  models 

•  Reduce  high-fidelity 
samples  needed  at 
optimization  iterations 
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Multifidelity  methods  lead  to  large 
computational  savings,  making 
optimization  under  uncertainty  tractable 


9 


MIT 


AEROASTR 


Design  under  uncertainty  of  an  acoustic  horn. 
Decision  variables:  horn  geometry,  b 
Uncertainty:  wavenumber,  wall  impedances 
Output  of  interest:  reflection  coefficient,  sr 

min  E[sr ]  +  J¥ar[sr] 

b 

Multifidelity  models: 

Finite  element  model  (35,895  states) 
Reduced  basis  model  (30  states) 

Multifidelity  approach: 

Control  variates 


Equivalent  number  of 
hi-fi  evaluations 


Regular  Monte 
Carlo 

Multifidelity 
Monte  Carlo 


44,343 

6,979 


(-84%) 


t  i  Uncertainty  propagation  using 
yjf  probabilistic  graphical  models 


Application:  Multiscale  aircraft  engine  disk  forging  problem 


Probabilistic  Graphical  model  framework 


Model  Reduction 

•  Localized  model 
reduction  scheme 


Structure  design 

Inference 

•  Decompose  the  joint 

•  Infer  the  marginal 

distribution  on  a  graph 

distribution  of  the 

using  localized  potentials 

response  (calculate  the 
statistics) 

Graph  Learning 

Learning  all  the 
unknown  parameters  in 
the  graph  (EM) 


Example:  Strength 


Mean 


ha 


MC  (N  =  4800) 


Std 


Graphical  Model 
(N  =  500) 


5 


Surrogate  model 


New 

Localized  Input 

Observation 

Model 

of  the  input 

Reduction 

Inference 

Infer  the 

conditional 

distribution 
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i  t  Ultra-Scalable  Algorithms  for  Large-Scale 
Uncertainty  Quantification  in  Inverse  Wave 

Propagation 

G.  Biros,  T.  Bui-Thanh,  L.  Demkowicz,  O.  Ghattas,  J.  T.  Oden,  G.  Stadler  (UT-Austin) 

•  Problem:  Address  inverse  problem  (aka  calibration  or  parameter  estimation)  in  the 
framework  of  Bayesian  inference:  given  a  model  with  uncertain  parameters  and 
noisy  observations  of  model  outputs,  estimate  the  model  parameters  and  quantify 
uncertainties  in  the  estimate 

•  Current  state-of-the-art  Markov  chain  Monte  Carlo  (MCMC)  methods  are  prohibitive 


•  The  issue  with  current  MCMC  methods  :  their  inability  to  exploit  problem  structure 


•  The  overarching  goal  is  to  overcome  the  “curse  of  dimensionality”  by  exploiting  the 
structure  of  the  posterior  probability  density  function  (pdf) 

-  Insight:  Hessian  operator  of  the  parameter-to-observable  map  defines  local  correlation 
structure  of  posterior  pdf,  and  can  be  used  to  accelerate  MCMC  sampling 

-  Construct  adjoint-based  low-rank  approximation  of  Hessian  to  make  tractable 

-  Develop  theory  at  infinite  dimensional  level,  ensure  mesh-independence  of  finite 
dimensional  discretization 


-  Develop  parallel  implementations  that  scale  to  the  limits  of  modern  supercomputers 
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Transformational 

Opportunities 


•  Tailor  theory  to  structure  of  inverse  wave  propagation  problems 


-  Estimate  heterogeneous  material  properties  from  scattered  waves 


•  Preliminary  results  demonstrate  1000X  speedups  on  small  problems 
relative  to  conventional  MCMC  methods;  permit  solution  of  large 
problems  that  were  previously  intractable 


•  Million-parameter  example  from  acoustic  wave  propagation:  infer 
global  heterogeneous  wave  speed  from  surface  observations  of 
seismic  waveforms 


•  1 .07  million  uncertain  parameters 

•  630M  states,  2400  time  steps 

•  Observations  only  in  N  Hemisphere 

•  1 00K  cores  on  Jaguar  Cray  XK6 

•  2000X  effective  dimension  reduction 

•  Top  row:  samples  from  the  prior 

•  Bottom  row:  samples  from  posterior 

•  Differences  between  rows  indicate 
knowledge  gained  from  data 
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Summary 


*  Continued  support  of  high-order  methods  in  multiscale 
and  multiphysics  modeling,  support  in  time-integration 
methods 

*  Continued  emphasis  on  UQ  and  V&V 

-  New  approached  for  reducing  the  curse  of 
dimensionality  (PDF  methods),  Optimal  Statistical 
Estimators,  Sampling  methods,  Design  under 
Uncertainty 

*  Algorithms  for  multi-core  platforms:  GPU  computing 

*  Emphasis  on  scalability  of  algorithms  for  ultra-parallel 
large  scale  computing 

*  Understanding  the  impact  of  geometric  discretization  on 
the  fidelity  of  the  analysis  and  computation 
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